The current invention is generally related to a user-controlled real-time computer animation for communicating with a viewer and is particularly related to a character image animated according to user voice and other inputs.
In the field of computer-graphics, a character has been animated for various purposes. Whether or not the character be a human, an animal or an object, computer scientists and computer-graphics animators have attempted to animate the character as if it is capable of communicating with a viewer. At the infancy of computer graphics, the character generally teaches or entertains the viewer in a non-interactive manner without responding to the viewer. As computer graphics has matured, the character had been animated in a slightly interactive manner. To support such an interaction between the character and the viewer, the character image must be animated on a real-time basis.
Despite significant advances in hardware and software, real-time character animation is still a difficult task. Among various images to be animated, character, human or otherwise, generally requires complex calculations at a high speed for rendering a large number of image frames. In particular, to communicate with a viewer in an interactive manner, the character animation must be able to synchronize its lip movement with an audio output as well as to express complex emotions. To accommodate such complex high-speed calculations, an expensive animation system including a high-performance processor is necessary. In addition, the complex input sub-system for inputting various information such as lip movements, limb movements and facial expressions are also necessary.
In the attempts to solve the above-described problems, the VACTOR(trademark) system, includes a high-performance three-dimensional rendering unit along with a complex input subsystem. An character image is rendered on a real-time basis based upon the inputs generated by a special sensor gear that a performer wears. The special sensor gear includes a position sensor placed around the lips of the performer, and certain exaggerated mouth movements generate desired mouth position signals. The performer also wears another set of position sensors on limbs for signaling certain limb movements. However, it has been reported that these position sensor gears are not ergonomically designed and requires a substantial amount of training to generate desired signals. Furthermore, the cost of the VACTOR(trademark) system is beyond the reach of most commercial enterprises and let alone individual consumers.
On the other hand, certain prior art two-dimensional animation systems do not generally require the above-described complex hardware and software and are usually affordable for the lack of realistic animation. For example, a two-dimensional character image is animated based upon the presence or the absence of a voice input. In this simplistic system, the mouth is animated open and closed during the voice input, and the animation is terminated when there is no more voice input. To animate the mouth, animation frames depicting the open mouth and the closed mouth is stored in an animation database, and upon the receipt of the voice input, an animation generator outputs the above-described animation frames to approximate the mouth movement in a crude manner. In other words, since this system has the same monotonous open and close mouth movement in response to various voice patterns, the animated character image fails to appear interactive. Furthermore, the image character generally fails to produce facial expressions.
As described above, the prior attempts have not yet solved the cost performance problem for a real-time animation system capable of generating a lively character image for communicating with a viewer using generally available personal computers such as IBM compatible or MacIntosh. A method and a system of generating a real-time yet lively character image on a widely available platform would substantially improve a cost performance relation that the above-described prior attempts had failed.
The animation system satisfying the above-described cost performance relation has a wide variety of application fields. In addition to traditional instructional and entertainment applications, for example, such a character animation system may be used to promote products at trade shows, author a short animation for various uses such as games and broadcasting, as well as to interface an end user. The image character animation may be authored in advance of the use or may be generated in response to a viewer response in an interactive manner.
To solve the above and other problems, according to a first aspect of the current invention, a computer graphics system for generating an animation sequence for a character in response to an input, includes an animation database unit for storing predetermined animation frames of the character, a sequence of the animation frames generating movements associated with communication; an animation status storage unit for storing a predetermined set of last input parameters; an input analyzer connected to the animation status storage unit for analyzing the input so as to determine the predetermined set of current input parameters in relation to the last input parameters, the input analyzer updating the last input parameters by the current input parameters; and an animation generator connected to the input analyzer for selecting a sequence of the animation frames based upon the current parameters so as to generate the animation sequence of the character.
According to a second aspect of the current invention, a computer graphics system for generating an animation sequence for a character in response to an input, includes an animation database unit for storing predetermined animation frames of the character, a sequence of the animation frames generating movements associated with communication; a character profile storage unit for storing a predetermined set of character profile parameters, the character profile parameters characterizing the character during the communication; an input analyzer connected to the character profile storage unit for analyzing the input so as to determine a predetermined set of current input parameters in relation to the character profile parameters; and an animation generator connected to the input analyzer for selecting a sequence of the animation frames based upon the current parameters so as to generate the animation sequence of the character.
According to a third aspect of the current invention, a computer graphics system for generating an animation sequence for a character in response to a voice input, includes an animation database unit for storing predetermined animation frames of the character, a sequence of the animation frames generating movements associated with oral communication, the movements including lip movements which approximate those of human speech; an input analyzer unit for analyzing the voice input so as to determine a predetermined set of input parameter values; and an animation generator connected to the input analyzer for selecting a sequence of the animation frames based upon the input parameter values so as to generate the movements of the character associated with the voice input.
According to a fourth aspect of the current invention, a virtual puppet system for animating a character on the fly on a display based upon an input, includes a facial expression unit for selecting a facial expression from a predetermined set of expressions based upon the input, the facial expression unit generating a facial expression signal; a facial orientation unit for selecting an orientation of a face of the character from a predetermined set of orientations with respect to the display based upon the input, the facial orientation unit generating a facial orientation signal; an input voice analyzing unit for analyzing the input for determining an input voice signal; and an animation generation unit connected to the facial expression unit, the facial orientation unit and the input voice analyzing unit for generating an animation sequence of the character based upon the facial expression signal, the facial orientation signal and the input voice signal, the facial expression signal, the facial orientation signal, the facial expression and the input voice signal being defined as current parameters.
According to a fifth aspect of the current invention, a virtual puppet system for animating a character on the fly to communicate with an audience, includes a character presentation unit for displaying the character to the audience; an audience response surveying unit for surveying an audience response to the character presented by the character presentation unit; an operator console having a voice input unit for generating an input voice signal based upon a voice input of an operator, a keypad for generating an expression signal so as to specify a character animation sequence; and an animation generating unit connected to the operator console and the character presentation unit for generating the animation sequence of the character based upon the input voice signal and the expression signal.
According to a sixth aspect of the current invention, an input voice analyzing unit for analyzing a digitized voice input, includes a wave processing unit for generating a plurality of frequency shifted waves based upon the digitized voice input, the wave processing unit adding the frequency shifted waves so as to generate an enhanced wave; a wave analysis unit connected to the wave processing unit for analyzing the enhanced wave so as to determine predetermined intermediate parameters; and a voice parameter generation unit connected to the wave analysis unit for generating a voice parameter set based upon the intermediate parameters, the voice parameter set indicating an increased sensitivity level for detecting a volume change of the voice input over a unit time.
According to a seventh aspect of the current invention, a method of generating an animation sequence for a character in response to an input, includes the steps of a) storing predetermined animation frames of the character, a sequence of the animation frames generating movements associated with communication; b) analyzing the input so as to determine the predetermined set of current input parameters in relation to the last input parameters, c) updating last input parameters by the current input parameters; and d) selecting a sequence of the animation frames based upon the current parameters so as to generate the animation sequence of the character.
According to an eighth aspect of the current invention, a method of generating an animation sequence for a character in response to an input, includes the steps of a) storing predetermined animation frames of the character, a sequence of the animation frames generating movements associated with communication; b) storing a predetermined set of character profile parameters, the character profile parameters characterizing the character during the communication; c) analyzing the input so as to determine a predetermined set of current input parameters in relation to the character profile parameters; and d) selecting a sequence of the animation frames based upon the current parameters so as to generate the animation sequence of the character.
According to a ninth aspect of the current invention, a method of generating an animation sequence for a character in response to a voice input, includes the steps of a) storing predetermined animation frames of the character, a sequence of the animation frames generating movements associated with oral communication, the movements including lip movements which approximate those of human speech; b) analyzing the voice input so as to determine a predetermined set of input parameter values; and c) selecting a sequence of the animation frames based upon the input parameter values so as to generate the movements of the character associated with the voice input.
According to a tenth aspect of the current invention, a method of animating a character on the fly on a display based upon an input, includes the steps of a) selecting a facial expression from a predetermined set of expressions based upon the input; b) selecting an orientation of a face of the character from a predetermined set of orientations with respect to the display based upon the input; c) analyzing the input for determining an input voice parameter; and d) generating an animation sequence of the character based upon the selected facial expression, the selected orientation and the input voice parameter, the selected facial orientation, the selected facial expression and the input voice parameter being defined as current parameters.
According to an eleventh aspect of the current invention, a method of animating a character on the fly to communicate with an audience, comprising the steps of a) displaying the character to the audience; b) surveying an audience response to the character as displayed in the step a); c) generating an input voice signal based upon a voice input of an operator and an expression signal so as to specify a character animation sequence; and d) generating the animation sequence of the character based upon the input voice signal and the expression signal.
According to a twelfth aspect of the current invention, a method of analyzing a digitized voice input, includes the steps of a) generating a plurality of frequency shifted waves based upon the digitized voice input; b) adding the frequency shifted waves so as to generate an enhanced wave; c) analyzing the enhanced wave so as to determine predetermined intermediate parameters; and d) generating a voice parameter set based upon the intermediate parameters, the voice parameter set indicating an increased sensitivity level for detecting a volume change of the voice input over a unit time.
These and various other advantages and features of novelty which characterize the invention are pointed out with particularity in the claims annexed hereto and forming a part hereof. However, for a better understanding of the invention, its advantages, and the objects obtained by its use, reference should be made to the drawings which form a further part hereof, and to the accompanying descriptive matter, in which there is illustrated and described a preferred embodiment of the invention.